The invention relates to a method for generating a large static image M(n), such as a sprite or a mosaic, from a video sequence including successive video objects VOs, said method comprising the steps of:
(A) estimating motion parameters related to the current video object V0(n) of the sequence, with respect to the previously generated static image M(nxe2x88x921);
(B) warping said current video object V0(n), on the basis of said estimated motion parameters;
(C) blending the warped video object WV0(n) thus obtained with the previously generated static image M(nxe2x88x921); and to a corresponding device. This invention may be useful in relation with the MPEG-4 and MPEG-7 standards.
The MPEG-7 standard has for object to standardize within some years generic ways to describe multimedia content, in view of a fast and efficient retrieval of data on the basis of various types of features such as text, color, texture, motion and semantic content. In this context, a mosaic can play a useful role. As indeed explained for instance in the article xe2x80x9cEfficient representations of video sequences and their applicationsxe2x80x9d, M. Irani and al., Signal Processing: Image Communication, vol.8, 1996, pp.327-351, a mosaic image is constructed from all frames in a scene sequence, giving a panoramic view of said scene. It is possible to extract for instance from this panoramic view the main features of the sequence, such as chrominance or luminance histograms, objects shapes, global motion parameters, and so on.
The definition of a mosaic may be compared to that of a sprite, as used in the context of the MPEG-4 standard. As described for instance in the case of a sprite in the document WO 98/59497, which can be also applied to the case of a mosaic, three main steps may compose a sprite or mosaic generation (in the following, the generic word xe2x80x9cstatic imagexe2x80x9d will therefore be used in place of sprite or mosaic). A motion estimation step is first provided, in order to find the motion parameters that allow to merge correctly a current frame F(n) with the static image M(nxe2x88x921) already composed of the previous frames F(1), F(2), . . . , F(nxe2x88x921). The inverse parameters are then computed, so that the current frame may be compensated in their direction; this second step is also called warping. The warped current frame F(n) is finally blended with M(nxe2x88x921) in order to form a new accreted static image M(n), with which the next incoming frame F(n+1) will be merged, and so on.
It is an object of the invention to propose a static image generation method with an increased robustness to outliers.
To this end, the invention relates to a method such as described in the introductory paragraph of the description and which is moreover characterized in that:
(1) said method also comprises, between said warping and blending steps, an additional step for computing, for each picture element of the current video object V0(n), a weighting coefficient wWF(n)[x,y] correlated to the error between the warped video object WV0(n) and the static image M(nxe2x88x921) at each picture element [x,y];
(2) the blending step provided for determining the newly generated static image M(n) takes into account said weighting coefficients according to the following blending formula, used for each pixel [x,y]:             M      ⁡              (        n        )              ⁡          [              x        ,        y            ]        =                                          w                          M              ⁡                              (                                  n                  -                  1                                )                                              ⁡                      [                          x              ,              y                        ]                          ·                              M            ⁢                          (                              n                -                1                            )                                ⁡                      [                          x              ,              y                        ]                              +                                    w                          WF              ⁡                              (                n                )                                              ⁡                      [                          x              ,              y                        ]                          ·                              WF            ⁢                          (              n              )                                ⁡                      [                          x              ,              y                        ]                                              w                  M          ⁡                      (                          n              -              1                        )                              +              w                  WF          ⁡                      (            n            )                              
xe2x80x83where the definitions of the terms are the following:
(a) n greater than 0
(b) whatever (x,y), wM(0)=0;
(c) whatever (x,y), wWF(n)[x,y]=                    w                  WF          ⁢                      (            n            )                              ⁡              [                  x          ,          y                ]              =                  1                  r          ⁢                      (                          x              ,              y                        )                              ⁢              ⅆ                  ⅆ          r                    ⁢              ρ        ⁢                  (                      r            ⁢                          (                              x                ,                y                            )                                )                      ;
(d) wM(n)=wM(nxe2x88x921)+wWF(n).
It is another object of the invention to propose a static image generation device for carrying out said method.
To this end, the invention relates to a device for generating a large static image M(n), such as a sprite or a mosaic, from a video sequence including successive video objects VOs, said device comprising:
(A) a motion estimation circuit, provided for estimating a motion information related to the relative motion between the current video object V0(n) of the sequence and the previously generated static image M(nxe2x88x921);
(B) a warping circuit, provided for defining on the basis of said current video object and said motion information a warped video object WVO(n);
(C) a blending circuit, provided for updating the static image, said previously generated static image M(nxe2x88x921) being replaced by the new one M(n) generated by blending said previously generated static image M(nxe2x88x921) and said warped video object; characterized in that:
(1) said device also comprises a pixel-based weighting circuit, receiving the current video object V0(n) and the associated motion information and computing for each picture element of said current video object a weighting coefficient that depends on the error between the warped video object WVO(n) and the previously generated static image M(nxe2x88x921) at each picture element [x,y];
(2) the blending formula used in the blending circuit for determining the newly generated static image M(n) takes into account the computed weighting coefficients according to the following blending formula, used for each pixel [x,y]:             M      ⁡              (        n        )              ⁡          [              x        ,        y            ]        =                                          w                          M              ⁡                              (                                  n                  -                  1                                )                                              ⁡                      [                          x              ,              y                        ]                          ·                              M            ⁢                          (                              n                -                1                            )                                ⁡                      [                          x              ,              y                        ]                              +                                    w                          WF              ⁡                              (                n                )                                              ⁡                      [                          x              ,              y                        ]                          ·                              WF            ⁢                          (              n              )                                ⁡                      [                          x              ,              y                        ]                                              w                  M          ⁡                      (                          n              -              1                        )                              +              w                  WF          ⁡                      (            n            )                              
xe2x80x83where the definitions of the terms are the following:
(a) n greater than 0
(b) whatever (x,y), wM(0)=0;
(c) whatever (x,y),                     W                  WF          ⁡                      (            n            )                              ⁡              [                  x          ,          y                ]              =                  1                  r          ⁡                      (                          x              ,              y                        )                              ⁢              ⅆ                  ⅆ          r                    ⁢      ρ      ⁢              xe2x80x83            ⁢              (                  r          ⁡                      (                          x              ,              y                        )                          )              ;
(d) wM(n)=wM(nxe2x88x921)+wWF(n).
The advantageous principle of the proposed method and device is that it is possible to accumulate confidence in the weight static image, in order not to take into account picture elements with low weight (i.e. outliers), which is carried out by summing for each picture element in the weight static image weights of the same picture element.